Methods and systems for enhanced scene perception using vehicle platoon

ABSTRACT

A vehicle includes one or more sensors configured to obtain raw data related to a scene, one or more processors, and machine readable instructions stored in one or more memory modules. The one machine readable instructions, when executed by the one or more processors, cause the vehicle to: process the raw data with a first neural network stored in the one or more memory modules to obtain a first prediction about the scene, transmit the raw data to a computing device external to the vehicle, receive a second prediction about the scene from the computing device in response to transmitting the raw data to the computing device, and determine an updated prediction about the scene based on a combination of the first prediction and the second prediction.

TECHNICAL FIELD

The present specification relates to scene perception using differentneural networks stored in different vehicles of a vehicle platoon.

BACKGROUND

A vehicle platoon is a group of vehicles that can travel very closelytogether. Each vehicle communicates with other vehicles in the vehicleplatoon. A lead vehicle controls the speed and direction, and allfollowing vehicles respond to the lead vehicle's movement. In thevehicle platoon, following vehicles rarely contribute to the drivingperformance of the lead vehicle.

Accordingly, a need exists for providing a method and system forutilizing resources of following vehicles in a vehicle platoon toenhance driving performance of the overall vehicle platoon.

SUMMARY

The present disclosure provides systems and methods for predicting andclassifying objects external to a vehicle platoon using different neuralnetworks stored in different vehicles of the vehicle platoon.

In one embodiment, a vehicle includes one or more sensors configured toobtain raw data related to a scene, one or more processors, and machinereadable instructions stored in one or more memory modules. The machinereadable instructions, when executed by the one or more processors,cause the vehicle to: process the raw data with a first neural networkstored in the one or more memory modules to obtain a first predictionabout the scene, transmit the raw data to a computing device external tothe vehicle, receive a second prediction about the scene from thecomputing device in response to transmitting the raw data to thecomputing device, and determine an updated prediction about the scenebased on a combination of the first prediction and the secondprediction.

In another embodiment, a vehicle platoon system includes a lead vehicleand a following vehicle. The lead vehicle includes one or more sensorsconfigured to obtain raw data related to a scene and a first controllerconfigured to process the raw data with a first neural network to obtaina first prediction about the scene. The following vehicle includes asecond controller configured to process the raw data received from thelead vehicle with a second neural network to obtain a second prediction.The first controller is configured to transmit the raw data to thefollowing vehicle, receive the second prediction about the scene fromthe following vehicle in response to transmitting the raw data to thefollowing vehicle, and determine an updated prediction based on acombination of the first prediction and the second prediction.

In yet another embodiment, a method includes obtaining raw data relatedto a scene using one or more sensors of a lead vehicle, processing theraw data with a first neural network stored in the lead vehicle toobtain a first prediction about the scene, transmitting the raw data toa following vehicle, receiving a second prediction about the scene fromthe following vehicle in response to transmitting the raw data to thefollowing vehicle, and determining an updated prediction about the scenebased on a combination of the first prediction and the secondprediction.

These and additional features provided by the embodiments of the presentdisclosure will be more fully understood in view of the followingdetailed description, in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments set forth in the drawings are illustrative and exemplaryin nature and not intended to limit the disclosure. The followingdetailed description of the illustrative embodiments can be understoodwhen read in conjunction with the following drawings, where likestructure is indicated with like reference numerals and in which:

FIG. 1 schematically depicts a vehicle platoon system that perceives ascene, for example, application of detecting and analyzing interestingelements in a scene, using different neural networks stored in differentvehicles, according to one or more embodiments shown and describedherein;

FIG. 2 depicts a schematic diagram for a system that perceives a scenewith or without external objects using different neural networks storedin different vehicles, according to one or more embodiments shown anddescribed herein;

FIG. 3 depicts a flowchart for perceiving a scene with or withoutexternal objects using different neural networks stored in differentvehicles, according to one or more embodiments shown and describedherein;

FIG. 4 depicts combining prediction results from different neuralnetworks, according to one or more embodiments shown and describedherein;

FIG. 5 depicts a flowchart for implementing a machine learning method,according to one or more embodiments shown and described herein;

FIG. 6 depicts detecting ground truth by a lead vehicle, according toanother embodiment shown and described herein; and

FIG. 7 depicts a combination of vehicles and an edge device thatperceives a scene with or without external objects using differentneural networks stored in different vehicles and the edge device,according to another embodiment shown and described herein.

DETAILED DESCRIPTION

The embodiments disclosed herein include systems and methods forperceiving scenes external to a vehicle platoon using different neuralnetworks stored in different vehicles of the vehicle platoon. Referringto FIGS. 1 and 2, a vehicle platoon system 100 includes a lead vehicle102 and following vehicles 104 and 106. The lead vehicle 102 includesone or more sensors configured to obtain raw data, e.g., an image 114 ofa scene with or without an object external to the vehicle platoon system100. The lead vehicle 102 includes a controller configured to processthe raw data with a neural network 103 to obtain a first predictionabout the scene. The prediction about the scene may include segmentingthe scene, detecting one or more objects in the scene, and/orclassifying one or more objects in the scene. The following vehicle 104includes a controller configured to process the raw data received fromthe lead vehicle 102 with a neural network 105 to obtain a secondprediction 124 about the scene. The neural network 105 is different formthe neural network 103. For example, the parameters, nodes and/or layersof the neural network 105 may be different from the parameters, nodesand/or layers of the neural network 103. The controller of the leadvehicle 102 transmits the raw data to the following vehicle 104,receives the second prediction 124 about the scene from the followingvehicle 104 in response to transmitting the raw data to the followingvehicle 104, and determines an updated prediction based on a combinationof the first prediction and the second prediction 124. The lead vehicle102 may also receive a third prediction 126 about the scene made by theneural network 107 of the following vehicle 106, and determine anupdated prediction based on a combination of the first prediction, thesecond prediction 124, and the third prediction 126.

According to the present disclosure, a vehicle platoon includes a leadvehicle and one or more following vehicles. The one or more followingvehicles may have a relatively easier driving environment as compared tothe lead vehicle. The following vehicles may only be required to stay ina lane and maintain a certain distance from the vehicle ahead. As aconsequence, the following vehicles may turn off or reduce use of somesensors (e.g., long distance radar sensors, Lidar sensors, and somecameras) and/or slow down or stop processing tasks (e.g.,computationally intensive neural network execution tasks related to theparticular vehicle) and operated mainly utilizing radar sensors and V2Xcommunication. The saved processing power of the following vehicles maybe redirected to help improve the neural network performance of the leadvehicle. Specifically, each of the following vehicles may receive rawdata from the lead vehicle and process the raw data using its own neuralnetwork that is different from the neural network of the lead vehicle.The predictions by the neural networks of the following vehicles may betransmitted to the lead vehicle. The lead vehicle may combine itspredictions made by the neural network of the lead vehicle and thepredictions made by the neural networks of the following vehicles. Thecombined predictions may hence overall performance of the vehicleplatoon because the combined prediction may enhance the accuracy of theprediction by the lead vehicle. For example, the combined prediction mayprevent erroneous prediction by the lead vehicle due to errors insensors, the neural network, or any other processing error.

FIG. 1 schematically depicts a vehicle platoon system that perceives ascene, for example, detecting and classifying external objects, usingdifferent neural networks stored in different vehicles, according to oneor more embodiments shown and described herein.

In embodiments, a vehicle platoon system 100 may include a plurality ofvehicles including a lead vehicle 102 and following vehicles 104 and106. While FIG. 1 illustrates two following vehicles 104 and 106, thevehicle platoon system 100 may include more than or less than twofollowing vehicles. The vehicle platoon system 100 may communicate witha server 160. The server 160 may be a remote server such as a cloudserver. In some embodiments, the server 160 may be a local serverincluding, but not limited to, a roadside unit, an edge server, and thelike.

Each of the lead vehicle 102 and the following vehicles 104 and 106 maybe a vehicle including an automobile or any other passenger ornon-passenger vehicle such as, for example, a terrestrial, aquatic,and/or airborne vehicle. In some embodiment, one or more of the leadvehicle 102 and the following vehicles 104 and 106 may be an unmannedaerial vehicle (UAV), commonly known as a drone.

One or more of the lead vehicle 102 and the following vehicles 104 and106 may be autonomous and connected vehicles, each of which navigatesits environment with limited human input or without human input. Thelead vehicle 102 and the following vehicles 104 and 106 are equippedwith internet access and share data with other devices both inside andoutside the lead vehicle 102 and the following vehicles 104 and 106. Thelead vehicle 102 and the following vehicles 104 and 106 may communicatewith the server 160. The server 160 may communicate with vehicles in anarea covered by the server 160. The server 160 may communicate withother servers that cover different areas. The server 160 may communicatewith a remote server and transmit information collected by the server160 to the remote server.

The lead vehicle 102 and the following vehicles 104 and 106 form avehicle platoon. A vehicle platoon is a group of vehicles that cantravel very closely together. Each vehicle communicates with othervehicles in the platoon. The lead vehicle 102 controls the speed anddirection, and the following vehicles 104 and 106 respond to the leadvehicle's movement.

In embodiments, each of the lead vehicle 102 and the following vehicles104 and 106 may include a neural network for interpreting a scene, forexample, segmenting the scene, detecting and/or classifying objects. Forexample, the lead vehicle 102 includes a neural network NN1 103, thefollowing vehicle 104 includes a neural network NN2 105, and thefollowing vehicle 106 includes a neural network NN3 107. The neuralnetworks 103, 105, and 107 may include different layers, nodes, and/orparameters such that the neural networks 103, 105 and 107 may outputdifferent data with respect to the same input.

In some embodiments, the server 160 may transmit different neuralnetworks to the vehicles 102, 104, and 106, respectively. For example,the server 160 may transmit the neural network NN1 103 to the leadvehicle 102, transmit the neural network NN2 105 to the followingvehicle 104, and transmit the neural network NN3 107 to the followingvehicle 106. In some embodiments, the server 160 may transmit thedifferent neural networks when the vehicles 102, 104, and 106 form avehicle platoon. For example, when the vehicles 102, 104, and 106 formthe vehicle platoon system 100, the vehicle platoon system 100 transmitsa notification to the server 160 that the vehicles 102, 104, and 106formed the vehicle platoon system 100. In response, the server 160 maytransmit and assign different neural networks 103, 105, 107 to vehicles102, 104, and 106, respectively.

In some embodiments, each of the lead vehicle 102 and the followingvehicles 104 and 106 may store a plurality of neural networks includingneural networks 103, 105, and 107. The lead vehicle 102 may select oneof the plurality of neural networks 103, 105, and 107 based on variousfactors including road conditions, a type of a road, a vehicle location,the status of a vehicle in a platoon (e.g., a lead vehicle or afollowing vehicle), time of the day, weather, and the like. Once thelead vehicle 102 selects the neural network 103 as a current neuralnetwork, then the lead vehicle 102 may transmit the information aboutthe neural network 103 to the following vehicles 104 and 106. Inresponse, each of the following vehicles 104 and 106 may select a neuralnetwork that is different from the neural network 103 of the leadvehicle 102. For example, the following vehicle 104 may select theneural network 105 as its current neural network, and the followingvehicle 106 may select the neural network 107 as its current neuralnetwork.

The following vehicles 104 and 106 may only be required to stay in alane and maintain a certain distance from the lead vehicle 102. As aconsequence, the following vehicles 104 and 106 may turn off or reduceuse of some sensors (e.g., long distance radar sensors, Lidar sensors,and some cameras) and/or slow down or stop processing tasks (e.g.,computationally intensive neural network execution tasks related to aparticular vehicle) and operate mainly utilizing radar sensors andvehicle-to-vehicle (V2V) or vehicle-to-everything (V2X) communication.

The saved processing power of the following vehicles 104 and 106 may beredirected to help improve the neural network performance of the leadvehicle 102 and hence overall performance of the vehicle platoon system100. In embodiments, the lead vehicle 102 may obtain raw data related toa scene including an object 130 or an object 140 from a distance asillustrated in FIG. 1. For example, the lead vehicle 102 may capture animage of the object 130 using one or more sensors such as a forwardfacing camera, radar, and the like. The lead vehicle 102 may process theraw data to detect the object 130. Then, the lead vehicle 102 may obtaina first prediction about the scene including the object 130 with theneural network 103 stored in the lead vehicle 102. The prediction may bea predicted classification of an object. The lead vehicle 102 maytransmit the raw data to the following vehicles 104 and 106. Thefollowing vehicle 104 may process the received raw data and detect theobject 130. Then, the following vehicle 104 may obtain a secondprediction 124 about the scene including the object 130 with the neuralnetwork 105 stored in the following vehicle 104. The following vehicle104 transmits the second prediction 124 to the lead vehicle 102.Similarly, the following vehicle 106 may process the received raw dataand detect the object 130. Then, the following vehicle 106 may obtain athird prediction 126 about the object 130 with the neural network 107stored in the following vehicle 106. The following vehicle 106 transmitsthe third prediction 126 to the lead vehicle 102. The lead vehicle 102may determine an updated prediction about the object 130 based on acombination of the first prediction, the second prediction 124, and thethird prediction 126. Combining the predictions from multiple neuralnetworks adds a bias that in turn counters the variance of a singletrained neural network model. Because the lead vehicle 102 and thefollowing vehicles 104 and 106 have different neural networks, combiningtheir outcomes from different neural networks is expected to improve theperception performance. The details about combining the predictions willbe described below with reference to FIGS. 3 through 6.

In some embodiments, the lead vehicle 102 may process the raw data andfind no object in the scene. The lead vehicle 102 may transmit the rawdata to the following vehicles 104 and 106. The following vehicle 104may process the received raw data and find no object in the scene, andreturn a message that no object is identified in the scene to the leadvehicle 102. Similarly, the following vehicle 106 may process thereceived raw data and find no object in the scene, and return a messagethat no object is identified in the scene to the lead vehicle 102.

FIG. 2 depicts a schematic diagram for a system that perceives a scenewith or without external objects using different neural networks storedin different vehicles, according to one or more embodiments shown anddescribed herein. The system for perceiving a scene with or withoutexternal objects includes a lead vehicle system 200, following vehiclesystems 220 and 260, and the server 160.

It is noted that, while the lead vehicle system 200 and the followingvehicle systems 220 and 260 are depicted in isolation, each of the leadvehicle system 200 and the following vehicle systems 220 and 260 may beincluded within a vehicle in some embodiments, for example, respectivelywithin each of the lead vehicle 102 and the following vehicles 104 and106 of FIG. 1. In embodiments in which each of the lead vehicle system200 and the following vehicle systems 220 and 260 is included within avehicle, the vehicle may be an automobile or any other passenger ornon-passenger vehicle such as, for example, a terrestrial, aquatic,and/or airborne vehicle. In some embodiments, the vehicle is anautonomous vehicle that navigates its environment with limited humaninput or without human input.

The lead vehicle system 200 includes one or more processors 202. Each ofthe one or more processors 202 may be any device capable of executingmachine readable and executable instructions. Accordingly, each of theone or more processors 202 may be a controller, an integrated circuit, amicrochip, a computer, or any other computing device. The one or moreprocessors 202 are coupled to a communication path 204 that providessignal interconnectivity between various modules of the system.Accordingly, the communication path 204 may communicatively couple anynumber of processors 202 with one another, and allow the modules coupledto the communication path 204 to operate in a distributed computingenvironment. Specifically, each of the modules may operate as a nodethat may send and/or receive data. As used herein, the term“communicatively coupled” means that coupled components are capable ofexchanging data signals with one another such as, for example,electrical signals via conductive medium, electromagnetic signals viaair, optical signals via optical waveguides, and the like.

Accordingly, the communication path 204 may be formed from any mediumthat is capable of transmitting a signal such as, for example,conductive wires, conductive traces, optical waveguides, or the like. Insome embodiments, the communication path 204 may facilitate thetransmission of wireless signals, such as WiFi, Bluetooth®, Near FieldCommunication (NFC) and the like. Moreover, the communication path 204may be formed from a combination of mediums capable of transmittingsignals. In one embodiment, the communication path 204 comprises acombination of conductive traces, conductive wires, connectors, andbuses that cooperate to permit the transmission of electrical datasignals to components such as processors, memories, sensors, inputdevices, output devices, and communication devices. Accordingly, thecommunication path 204 may comprise a vehicle bus, such as for example aLIN bus, a CAN bus, a VAN bus, and the like. Additionally, it is notedthat the term “signal” means a waveform (e.g., electrical, optical,magnetic, mechanical or electromagnetic), such as DC, AC,sinusoidal-wave, triangular-wave, square-wave, vibration, and the like,capable of traveling through a medium.

The lead vehicle system 200 includes one or more memory modules 206coupled to the communication path 204. The one or more memory modules206 may comprise RAM, ROM, flash memories, hard drives, or any devicecapable of storing machine readable and executable instructions suchthat the machine readable and executable instructions can be accessed bythe one or more processors 202. The machine readable and executableinstructions may comprise logic or algorithm(s) written in anyprogramming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or5GL) such as, for example, machine language that may be directlyexecuted by the processor, or assembly language, object-orientedprogramming (OOP), scripting languages, microcode, etc., that may becompiled or assembled into machine readable and executable instructionsand stored on the one or more memory modules 206. Alternatively, themachine readable and executable instructions may be written in ahardware description language (HDL), such as logic implemented viaeither a field-programmable gate array (FPGA) configuration or anapplication-specific integrated circuit (ASIC), or their equivalents.Accordingly, the methods described herein may be implemented in anyconventional computer programming language, as pre-programmed hardwareelements, or as a combination of hardware and software components.

The one or more memory modules 206 may include one or more neuralnetworks including the neural network 103. The one or more memorymodules 206 may include machine readable instructions that, whenexecuted by the one or more processors 202, cause the lead vehiclesystem 200 to receive raw data from one or more sensors, process rawdata with the neural network 103 to obtain a first prediction about ascene, transmit the raw data to a computing device external to thevehicle, such as the following vehicle systems 220 and 260, receivepredictions about the scene from the following vehicle systems 220 and260, and determine an updated prediction about the scene based on acombination of the prediction by the lead vehicle system 200 and thepredictions by the following vehicle systems 220 and 260.

Referring still to FIG. 2, the lead vehicle system 200 comprises one ormore sensors 208. The one or more sensors 208 may be any device havingan array of sensing devices capable of detecting radiation in anultraviolet wavelength band, a visible light wavelength band, or aninfrared wavelength band. The one or more sensors 208 may have anyresolution. In some embodiments, one or more optical components, such asa mirror, fish-eye lens, or any other type of lens may be opticallycoupled to the one or more sensors 208. In embodiments described herein,the one or more sensors 208 may provide image data to the one or moreprocessors 202 or another component communicatively coupled to thecommunication path 204. The image data may include image data of a scenewith or without the object 130 or the object 140 in FIG. 1. In someembodiments, the one or more sensors 208 may also provide navigationsupport. That is, data captured by the one or more sensors 208 may beused to autonomously or semi-autonomously navigate the lead vehicle 102.

In some embodiments, the one or more sensors 208 include one or moreimaging sensors configured to operate in the visual and/or infraredspectrum to sense visual and/or infrared light. Additionally, while theparticular embodiments described herein are described with respect tohardware for sensing light in the visual and/or infrared spectrum, it isto be understood that other types of sensors are contemplated. Forexample, the systems described herein could include one or more LIDARsensors, radar sensors, sonar sensors, or other types of sensors andthat such data could be integrated into or supplement the datacollection described herein to develop a fuller real-time traffic image.Ranging sensors like radar may be used to obtain a rough depth and speedinformation for the view of the lead vehicle system 200. The leadvehicle system 200 may capture a scene with or without an object such asthe object 130 or the object 140 in FIG. 1 using one or more imagingsensors. The one or more processors 202 may process the captured imagewith the neural network 103 to detect the object 130 or the object 140.

In operation, the one or more sensors 208 capture image data andcommunicate the image data to the one or more processors 202 and/or toother systems communicatively coupled to the communication path 204. Theimage data may be received by the one or more processors 202, which mayprocess the image data using one or more image processing algorithms.Any known or yet-to-be developed video and image processing algorithmsmay be applied to the image data in order to identify an item orsituation. Example video and image processing algorithms include, butare not limited to, kernel-based tracking (such as, for example,mean-shift tracking) and contour processing algorithms. In general,video and image processing algorithms may detect objects and movementfrom sequential or individual frames of image data. One or more objectrecognition algorithms may be applied to the image data to extractobjects. Any known or yet-to-be-developed object recognition algorithmsmay be used to extract the objects or even optical characters and imagesfrom the image data. Example object recognition algorithms include, butare not limited to, scale-invariant feature transform (“SIFT”), speededup robust features (“SURF”), and edge-detection algorithms.

The lead vehicle system 200 comprises a satellite antenna 214 coupled tothe communication path 204 such that the communication path 204communicatively couples the satellite antenna 214 to other modules ofthe lead vehicle system 200. The satellite antenna 214 is configured toreceive signals from global positioning system satellites. Specifically,in one embodiment, the satellite antenna 214 includes one or moreconductive elements that interact with electromagnetic signalstransmitted by global positioning system satellites. The received signalis transformed into a data signal indicative of the location (e.g.,latitude and longitude) of the satellite antenna 214 or an objectpositioned near the satellite antenna 214, by the one or more processors202.

The lead vehicle system 200 comprises one or more vehicle sensors 212.Each of the one or more vehicle sensors 212 is coupled to thecommunication path 204 and communicatively coupled to the one or moreprocessors 202. The one or more vehicle sensors 212 may include one ormore motion sensors for detecting and measuring motion and changes inmotion of the vehicle. The motion sensors may include inertialmeasurement units. Each of the one or more motion sensors may includeone or more accelerometers and one or more gyroscopes. Each of the oneor more motion sensors transforms sensed physical movement of thevehicle into a signal indicative of an orientation, a rotation, avelocity, or an acceleration of the vehicle.

Still referring to FIG. 2, the lead vehicle system 200 comprises networkinterface hardware 216 for communicatively coupling the lead vehiclesystem 200 to the following vehicle systems 220 and 260 and/or theserver 160. The network interface hardware 216 can be communicativelycoupled to the communication path 204 and can be any device capable oftransmitting and/or receiving data via a network. Accordingly, thenetwork interface hardware 216 can include a communication transceiverfor sending and/or receiving any wired or wireless communication. Forexample, the network interface hardware 216 may include an antenna, amodem, LAN port, Wi-Fi card, WiMax card, mobile communications hardware,near-field communication hardware, satellite communication hardwareand/or any wired or wireless hardware for communicating with othernetworks and/or devices. In one embodiment, the network interfacehardware 216 includes hardware configured to operate in accordance withthe Bluetooth® wireless communication protocol. The network interfacehardware 216 of the lead vehicle system 200 may transmit its data to thefollowing vehicle systems 220 and 260 or the server 160. For example,the network interface hardware 216 of the lead vehicle system 200 maytransmit captured images generated by the lead vehicle system 200,vehicle data, location data, information about the neural network 103and the like to the following vehicle systems 220 and 260 or the server160.

The lead vehicle system 200 may connect with one or more externalvehicle systems (e.g., the following vehicle systems 220 and 260) and/orexternal processing devices (e.g., the server 160) via a directconnection. The direct connection may be a vehicle-to-vehicle connection(“V2V connection”) or a vehicle-to-everything connection (“V2Xconnection”). The V2V or V2X connection may be established using anysuitable wireless communication protocols discussed above. A connectionbetween vehicles may utilize sessions that are time-based and/orlocation-based. In embodiments, a connection between vehicles or betweena vehicle and an infrastructure element may utilize one or more networksto connect (e.g., the network 250), which may be in lieu of, or inaddition to, a direct connection (such as V2V or V2X) between thevehicles or between a vehicle and an infrastructure. By way ofnon-limiting example, vehicles may function as infrastructure nodes toform a mesh network and connect dynamically on an ad-hoc basis. In thisway, vehicles may enter and/or leave the network at will, such that themesh network may self-organize and self-modify over time. Othernon-limiting network examples include vehicles forming peer-to-peernetworks with other vehicles or utilizing centralized networks that relyupon certain vehicles and/or infrastructure elements. Still otherexamples include networks using centralized servers and other centralcomputing devices to store and/or relay information between vehicles.

Still referring to FIG. 2, the lead vehicle system 200 may becommunicatively coupled to the server 160 by the network 250. In oneembodiment, the network 250 may include one or more computer networks(e.g., a personal area network, a local area network, or a wide areanetwork), cellular networks, satellite networks and/or a globalpositioning system and combinations thereof. Accordingly, the leadvehicle system 200 can be communicatively coupled to the network 250 viaa wide area network, via a local area network, via a personal areanetwork, via a cellular network, via a satellite network, etc. Suitablelocal area networks may include wired Ethernet and/or wirelesstechnologies such as, for example, wireless fidelity (Wi-Fi). Suitablepersonal area networks may include wireless technologies such as, forexample, IrDA, Bluetooth®, Wireless USB, Z-Wave, ZigBee, and/or othernear field communication protocols. Suitable cellular networks include,but are not limited to, technologies such as LTE, WiMAX, UMTS, CDMA, andGSM.

Still referring to FIG. 2, the server 160 includes one or moreprocessors 242, one or more memory modules 246, network interfacehardware 248, and a communication path 244. The one or more processors242 may be a controller, an integrated circuit, a microchip, a computer,or any other computing device. The one or more memory modules 246 maycomprise RAM, ROM, flash memories, hard drives, or any device capable ofstoring machine readable and executable instructions such that themachine readable and executable instructions can be accessed by the oneor more processors 242. The communication path 244 may be similar to thecommunication path 204 in some embodiments. The one or more memorymodules 246 may include one or more neural networks and the networkinterface hardware 248 may transmit the one or more neural networks tothe lead vehicle system 200 and/or the following lead vehicle systems220 and 260 via the network 250.

Still referring to FIG. 2, the following vehicle system 220 includes oneor more processors 222, one or more memory modules 226, one or moresensors 228, one or more vehicle sensors 232, a satellite antenna 234,network interface hardware 236, and a communication path 224communicatively connected to the other components of the followingvehicle system 220. The components of the following vehicle system 220may be structurally similar to and have similar functions as thecorresponding components of the lead vehicle system 200 (e.g., the oneor more processors 222 corresponds to the one or more processors 202,the one or more memory modules 226 corresponds to the one or more memorymodules 206, the one or more sensors 228 corresponds to the one or moresensors 208, the one or more vehicle sensors 232 corresponds to the oneor more vehicle sensors 212, the satellite antenna 234 corresponds tothe satellite antenna 214, the network interface hardware 236corresponds to the network interface hardware 216, and the communicationpath 224 corresponds to the communication path 204). The one or morememory modules 226 may include one or more neural networks. The one ormore processors 222 may select one of the one or more neural networks,e.g., the neural network NN2 105, which is different from the neuralnetwork NN1 103 of the lead vehicle system 200. The parameters, nodesand/or layers of the neural network NN2 105 may be different from theparameters, nodes and/or layers of the neural network NN1 103.

Similarly, the following vehicle system 260 includes one or moreprocessors 262, one or more memory modules 266, one or more sensors 268,one or more vehicle sensors 272, a satellite antenna 274, networkinterface hardware 276, and a communication path 264 communicativelyconnected to the other components of the following vehicle system 260.The components of the following vehicle system 260 may be structurallysimilar to and have similar functions as the corresponding components ofthe lead vehicle system 200 (e.g., the one or more processors 262corresponds to the one or more processors 202, the one or more memorymodules 266 corresponds to the one or more memory modules 206, the oneor more sensors 268 corresponds to the one or more sensors 208, the oneor more vehicle sensors 272 corresponds to the one or more vehiclesensors 212, the satellite antenna 274 corresponds to the satelliteantenna 214, the network interface hardware 276 corresponds to thenetwork interface hardware 216, and the communication path 264corresponds to the communication path 204). The one or more memorymodules 266 may include one or more neural networks. The one or moreprocessors 262 may select one of the one or more neural networks, e.g.,the neural network NN3 107, which is different from the neural networkNN1 103 of the lead vehicle system 200. The parameters, nodes and/orlayers of the neural network NN3 107 may be different from theparameters, nodes and/or layers of the neural network NN1 103.

FIG. 3 depicts a flowchart for perceiving a scene with or withoutexternal objects using different neural networks stored in differentvehicles, according to one or more embodiments shown and describedherein.

In step 310, a lead vehicle may obtains raw data related to a scene withor without an object using one or more sensors of the lead vehicle. Inembodiments, by referring to FIGS. 1, 2, and 4, the lead vehicle system200 may obtain an image 114 of an external view or a scene, of the leadvehicle 102 from one or more sensors 208. The image 114 may include theobject 130 and/or the object 140. Ranging sensors such as radar sensorsmay be also used to determine a rough depth and speed information forthe external view or the scene.

Referring back to FIG. 3, in step 320, the lead vehicle may process theraw data with a first neural network stored in the lead vehicle toobtain a first prediction about the scene. The first prediction aboutthe scene may include segmenting the scene, detecting one or moreobjects in the scene, and/or classifying one or more objects in thescene. In embodiments, by referring to FIGS. 1, 2, and 4, the leadvehicle system 200 may process the image 114 using the neural network103 to detect objects in the image 114. In embodiments, the capturedimage 114 may be segmented out at the instance level. Any objectdetection algorithm may be used to detect objects in the captured image.For example, as shown in FIG. 4, the lead vehicle system 200 may segmentinstances 402 and 404 from the captured image 114. Then, the segmentedinstances 402 and 404 may be input to the neural network 103. The neuralnetwork 103 may be a convolutional neural network that extracts featuresfrom each segmented instance and classifies the features as one of aknown class with a probability. The neural network 103 may outputclassifications of the segmented instances 402 and 404 along withprobabilities as shown in a box 410. For example, as illustrated in FIG.4, the neural network 103 may predict the object 130 as a traffic conewith a probability of 90 percent. The neural network 103 may predict theobject 140 as a pedestrian with a probability of 60 percent, and as atree with a probability of 35 percent.

Referring back to FIG. 3, in step 330, the lead vehicle may transmit theraw data to a following vehicle. In embodiments, by referring to FIGS.1, 2, and 4, the lead vehicle system 200 may transmit the image 114captured by the lead vehicle 102 to following vehicles 104 and 106. Thefollowing vehicle system 220 of the following vehicle 104 may processthe image 114 using the neural network 105 to classify objects in theimage 114. In embodiments, the captured image 114 may be segmented outat the instance level. Any object detection algorithm may be used todetect objects in the captured image. For example, as shown in FIG. 4,the lead vehicle system 200 may segment instances 402 and 404 from thecaptured image 114. Then, the segmented instances 402 and 404 may beinput to the neural network 105. The neural network 105 may be aconvolutional neural network that extracts features from each segmentedinstance and classifies the features as one of a known class with aprobability. The neural network 105 may output classifications of thesegmented instances 402 and 404 along with probabilities as shown in abox 420. For example, as illustrated in FIG. 4, the neural network 105may classify the object 130 as a traffic cone with a probability of 88percent. The neural network 105 may classify the object 140 as apedestrian with a probability of 15 percent, and as a tree with aprobability of 80 percent.

Similarly, the following vehicle system 260 of the following vehicle 106may process the image 114 using the neural network 107 to classifyobjects in the image 114. In embodiments, the captured image 114 may besegmented out at the instance level. Any object detection algorithm maybe used to detect objects in the captured image. For example, as shownin FIG. 4, the lead vehicle system 200 may segment instances 402 and 404from the captured image 114. Then, the segmented instances 402 and 404may be input to the neural network 107. The neural network 107 may be aconvolutional neural network that extracts features from each segmentedinstance and classifies the features as one of a known class with aprobability. The neural network 107 may output classifications of thesegmented instances 402 and 404 along with probabilities as shown in abox 430. For example, as illustrated in FIG. 4, the neural network 107may classify the object 130 as a traffic cone with a probability of 89percent. The neural network 107 may classify the object 140 as apedestrian with a probability of 10 percent, and as a tree with aprobability of 85 percent.

Referring back to FIG. 3, in step 340, the lead vehicle may receive asecond prediction about the scene from the following vehicle in responseto transmitting the raw data to the following vehicle. The secondprediction about the scene may include segmenting the scene, detectingone or more objects in the scene, and/or classifying one or more objectsin the scene. In embodiments, by referring to FIGS. 1, 2, and 4, thelead vehicle system 200 may receive a second prediction made by thefollowing vehicle system 220. The second prediction may classify theobject 130 as a traffic cone with a probability of 88 percent andclassify the object 140 as a pedestrian with a probability of 15percent, and as a tree with a probability of 80 percent. The leadvehicle system 200 may also receive an additional prediction, e.g., athird prediction, made by the following vehicle system 260. The thirdprediction about the scene may include segmenting the scene, detectingone or more objects in the scene, and/or classifying one or more objectsin the scene. For example, the third prediction may classify the object130 as a traffic cone with a probability of 89 percent, and classify theobject 140 as a pedestrian with a probability of 10 percent, and as atree with a probability of 85 percent.

Referring back to FIG. 3, in step 350, the lead vehicle may determine anupdated prediction about the scene based on a combination of the firstprediction and the second prediction. Combining the outcomes fromdifferent neural networks may be implemented using various methods. Onemethod may be an equal averaging method that averages the predictions.In embodiments, by referring to FIGS. 2 and 4, the lead vehicle system200 may average prediction probabilities for each objects. For example,the lead vehicle system 200 may average the prediction probabilities forthe segmented instance 402 made by the neural network 103 of the leadvehicle 102, the neural network 105 of the following vehicle 104, andthe neural network 107 of the following vehicle 106. In this example,the average prediction for the segmented instance 402 is a traffic conewith a probability of 89 percent. Similarly, the lead vehicle system 200may average the probabilities for the segmented instance 404 made by theneural network 103 of the lead vehicle 102, the neural network 105 ofthe following vehicle 104, and the neural network 107 of the followingvehicle 106. In this example, the average prediction for the segmentedinstance 404 is a tree with a probability of 63 percent of a tree and apedestrian with a probability of 28 percent. In this example, while thelead vehicle 102 predicts the object 140 as a pedestrian with a higherprobability than a tree, the combination of the predictions made bythree different neural networks shows a different result than theprediction made solely by the lead vehicle 102.

In some embodiments, the first prediction, the second prediction, andthe third prediction may be prediction vectors. The lead vehicle system200 may determine the updated prediction about the object by averagingthe prediction vector of the first prediction, the prediction vector ofthe second prediction, and the prediction vector of the thirdprediction. Other methods may be used to combine the outcomes fromdifferent neural networks. For example, a machine learning method suchas a reinforcement learning method may be used. The outcome from theneural networks 103, 105, and 107 may be input to the box 440 which maybe a neural network whose parameters may be adopted based on thecomparison of the outcomes from the neural networks 103, 105, and 107and ground truth information obtained by the lead vehicle. The detailsof the machine learning method will be described in detail withreference to FIGS. 5 and 6 below.

In some embodiments, the lead vehicle system 200 may compare the firstprediction to the second prediction made by the following vehicle system220 and/or the third prediction made by the following vehicle system260. If the first prediction is significantly different from the secondprediction and/or the third prediction, the lead vehicle system 200 mayinstruct the lead vehicle 102 to opt out of the vehicle platoon system100. Similarly, the following vehicle system 220 may receive the firstprediction from the lead vehicle system 200 and the third predictionfrom the following vehicle system 260, and compare the second predictionto the first prediction made by the lead vehicle system 200 and/or thethird prediction made by the following vehicle system 260. If the secondprediction is significantly different from the first prediction and/orthe third prediction, the following vehicle system 220 may instruct thefollowing vehicle 104 to opt out of the vehicle platoon system 100.

FIG. 5 depicts a flowchart for implementing a machine learning method,according to one or more embodiments shown and described herein.

In step 510, a lead vehicle may obtain raw data related to a sceneincluding an object using one or more sensors of the lead vehicle. Inembodiments, by referring to FIGS. 1, 2 and 4, the lead vehicle system200 may obtain an image 114 of an external view or a scene of the leadvehicle 102 from one or more sensors 208. The image 114 may include theobject 130 and the object 140. Ranging sensors such as radar sensors maybe also used to determine a rough depth and speed information for theexternal view or the scene.

Referring back to FIG. 5, in step 520, a lead vehicle processes the rawdata with a first neural network stored in the lead vehicle to obtain afirst prediction about the scene. The first prediction about the scenemay include segmenting the scene, detecting one or more objects in thescene, and/or classifying one or more objects in the scene. Inembodiments, by referring to FIGS. 1, 2 and 4, the lead vehicle system200 may process the image 114 using the neural network 103 to detectobjects in the image 114. In embodiments, the captured image 114 may besegmented out at the instance level. Any object detection algorithm maybe used to detect objects in the captured image. For example, asillustrated in FIG. 4, the lead vehicle system 200 may segment instances402 and 404 from the captured image 114. Then, the segmented instances402 and 404 may be input to the neural network 103. The neural network103 may be a convolutional neural network that extracts features fromeach segmented instance and classifies the features as one of a knownclass with a probability. The neural network 103 may outputclassifications of the segmented instances 402 and 404 along withprobabilities. For example, as illustrated in FIG. 4, the neural network103 may predict the object 130 as a traffic cone with a probability of90 percent. The neural network 103 may predict the object 140 as apedestrian with a probability of 60 percent, and as a tree with aprobability of 35 percent.

Referring back to FIG. 5, in step 530, the lead vehicle may transmit theraw data to a following vehicle. In embodiments, by referring to FIGS.1, 2, and 4, the lead vehicle system 200 may transmit the image 114captured by the lead vehicle 102 to following vehicles 104 and 106. Thefollowing vehicle system 220 of the following vehicle 104 may processthe image 114 using the neural network 105 to classify objects in theimage 114. In embodiments, the captured image 114 may be segmented outat the instance level. Any object detection algorithm may be used todetect objects in the captured image. For example, as shown in FIG. 4,the lead vehicle system 200 may segment instances 402 and 404 from thecaptured image 114. Then, the segmented instances 402 and 404 may beinput to the neural network 105. The neural network 105 may be aconvolutional neural network that extracts features from each segmentedinstance and classifies the features as one of a known class with aprobability. The neural network 105 may output classifications of thesegmented instances 402 and 404 along with probabilities. For example,as illustrated in FIG. 4, the neural network 105 may classify the object130 as a traffic cone with a probability of 88 percent. The neuralnetwork 105 may classify the object 140 as a pedestrian with aprobability of 15 percent, and as a tree with a probability of 80percent.

Referring back to FIG. 5, in step 540, the lead vehicle may receive asecond prediction about the scene from the following vehicle in responseto transmitting the raw data to the following vehicle. The secondprediction about the scene may include segmenting the scene, detectingone or more objects in the scene, and/or classifying one or more objectsin the scene. In embodiments, by referring to FIGS. 1, 2, and 4, thelead vehicle system 200 may receive a second prediction made by thefollowing vehicle system 220. The second prediction may classify theobject 130 as a traffic cone with a probability of 88 percent andclassify the object 140 as a pedestrian with a probability of 15percent, and as a tree with a probability of 80 percent. The leadvehicle system 200 may also receive an additional prediction, e.g., athird prediction, made by the following vehicle system 260. The thirdprediction about the scene may include segmenting the scene, detectingone or more objects in the scene, and/or classifying one or more objectsin the scene. For example, the third prediction may classify the object130 as a traffic cone with a probability of 89 percent, and classify theobject 140 as a pedestrian with a probability of 10 percent, and as atree with a probability of 85 percent.

Referring back to FIG. 5, in step 550, the lead vehicle may obtainground truth information about the scene. In embodiments, by referringto FIG. 6, the lead vehicle 102 may obtain ground truth informationabout the objects 130 and 140 when the lead vehicle 102 is sufficientlyclose to the objects 130 and 140. For example, the lead vehicle 102 maycapture images of the objects 130 and 140 when the lead vehicle 102 isvery close to the objects 130 and 140 and obtain ground truth that theobject 130 is a tree and the object 140 is a traffic cone based on thecaptured images.

Referring back to FIG. 5, in step 560, the lead vehicle may update oneor more parameters of the first neural network based on a comparison ofthe ground truth information, the first prediction and the secondprediction. In embodiments, by referring to FIGS. 2 and 6, the leadvehicle 102 may compare the ground truth information, to the predictionmade by the neural network 103, the prediction made by the neuralnetwork 105, and the prediction made by the neural network 107. Based onthe comparison, the lead vehicle system 200 of the lead vehicle 102 mayupdate the one or more parameters of the neural network 103. In thisexample, the ground truth about the object 130 is a tree and the groundtruth about the object 140 is a traffic cone. For example, thepredictions made by the neural networks 103, 105, and 107 regarding theobjects 130 and 140 are shown in the table below.

Neural Network 103 Neural Network 105 Neural Network 107 Object 130Pedestrian: 60% Pedestrian: 15% Pedestrian: 10% Tree: 35% Tree: 80%Tree: 85% Object 140 Traffic Cone: 90% Traffic Cone: 88% Traffic Cone:89%

The predictions made by the neural networks 105 and 107 are moreaccurate than the prediction made by the neural network 103 regardingthe object 130 because the predictions made by the neural networks 105and 107 are closer to the ground truth than the prediction made by theneural network 103. Based on the comparison, the lead vehicle system 200of the lead vehicle 102 may update the one or more parameters of theneural network 103 based on the parameters of the neural network 105 orthe neural network 107. The lead vehicle system 200 of the lead vehicle102 may update the one or more parameters of the neural network 103 suchthat the neural network 103 may predict the object 130 as a tree with ahigher probability than 35 percent in response to receiving the image114 as an input.

As another example, the predictions made by the neural networks 103,105, and 107 regarding the objects 130 and 140 are shown in the tablebelow.

Neural Network 103 Neural Network 105 Neural Network 107 Object 130Pedestrian:  5% Pedestrian: 15% Pedestrian: 10% Tree: 90% Tree: 80%Tree: 85% Object 140 Traffic Cone: 90% Traffic Cone: 88% Traffic Cone:89%

In this case, the predictions made by the neural network 103 is moreaccurate than the predictions made by the neural networks 105 and 107regarding the objects 130 and 140 because the predictions made by theneural network 103 are closer to the ground truth than the predictionsby the neural networks 105 and 107. Based on the comparison, the leadvehicle system 200 of the lead vehicle 102 may update the one or moreparameters of the neural network 103. For example, the lead vehiclesystem 200 of the lead vehicle 102 may update the one or more parametersof the neural network 103 such that the neural network 103 may predictthe object 130 as a tree and the object as a traffic cone with anincreased probability (e.g., over 90 percent for the tree and over 90percent for the traffic cone) in response to receiving the image 114 asan input.

FIG. 7 depicts a combination of vehicles and an edge device that detectsand classifies external objects using different neural networks storedin different vehicles and the edge device, according to anotherembodiment shown and described herein.

In embodiments, a group of a vehicle 710, a vehicle 720, and an edgeserver 730 may constitute a temporary platoon similar to the vehicleplatoon system 100 in FIG. 1. The vehicle 710 and the vehicle 720 maytemporarily stop at a cross-section, and the edge server 730 may be aroad-side unit. The vehicle 710 may communicate with the vehicle 720 andthe edge server 730 via V2V or V2X communication. The vehicle 710 mayinclude a system similar to the lead vehicle system 200 in FIG. 2 andthe vehicle 720 may include a system similar to the following vehiclesystem 220 in FIG. 2. The vehicle 720 stops at an intersection. As aconsequence, the vehicle 720 may turn off or reduce use of some sensors(e.g., long distance radar sensors, Lidar sensors, and some cameras)and/or slow down or stop processing tasks (e.g., computationallyintensive neural network execution tasks related to a particularvehicle) and operate mainly utilizing radar sensors andvehicle-to-vehicle (V2V) or vehicle-to-everything (V2X) communication.

The vehicle 710 may obtain raw data related to a scene including anobject 740 using one or more sensors of the vehicle 710. In embodiments,by referring to FIGS. 2 and 7, the vehicle system 200 may obtain animage 714 of an external view or a scene of the vehicle 710 from one ormore sensors 208. The image 714 may include the object 740. Rangingsensors such as radar sensors may be also used to determine a roughdepth and speed information for the external view or the scene.

The vehicle 710 may process the image 714 using a neural network NN4 712stored in the vehicle 710 to classify the object in the image 714. Inembodiments, the captured image 714 may be segmented out at the instancelevel. Any object detection algorithm may be used to detect objects inthe captured image. For example, the vehicle 710 may segment instancesfrom the captured image 714. Then, the segmented instances may be inputto the neural network 712. The neural network 712 may be a convolutionalneural network that extracts features from each segmented instance andclassifies the features as one of a known class with a probability. Theneural network 712 may output classifications of segmented instancesalong with probabilities. For example, the neural network 712 maypredict the object 740 as a pothole with a probability of 30 percent andas an animal with a probability of 65 percent.

The vehicle 710 may transmit the image 714 captured by the vehicle 710to the vehicle 720 and the edge server 730. The vehicle 720 may processthe image 714 using a neural network NN5 722 stored in the vehicle 720to classify objects in the image 714. The neural network 722 isdifferent form the neural network 712. For example, the parameters,nodes and/or layers of the neural network 722 may be different from theparameters, nodes and/or layers of the neural network 712. The neuralnetwork 722 may be a convolutional neural network that extracts featuresfrom each segmented instance and classifies the features as one of aknown class with a probability. The neural network 722 may outputclassifications of segmented instances along with probabilities. Forexample, as illustrated in FIG. 7, the neural network 712 may predictthe object 740 as a pothole with a probability of 85 percent and as ananimal with a probability of 10 percent.

Similarly, the edge server 730 may process the image 714 using a neuralnetwork NN6 732 stored in the edge server 730 to classify objects in theimage 714. The neural network 732 is different form the neural network712. For example, the parameters, nodes and/or layers of the neuralnetwork 732 may be different from the parameters, nodes and/or layers ofthe neural network 712. In embodiments, the captured image 714 may besegmented out at the instance level. Any object detection algorithm maybe used to detect objects in the captured image. The neural network 732may output classifications of segmented instances along withprobabilities. For example, as illustrated in FIG. 7, the neural network732 may predict the object 740 as a pothole with a probability of 80percent and as an animal with a probability of 15 percent.

Then, the vehicle 710 may receive predictions about the object 740 fromthe vehicle 720 and the edge server 730. In embodiments, by referring toFIG. 7, the vehicle 710 may receive a prediction 724 made by the vehicle720 and a prediction 734 made by the edge server 730. The vehicle 710may determine an updated prediction about the object 740 based on acombination of the prediction made by the vehicle 710, the prediction724 made by the vehicle 720, and the prediction 734 made by the edgeserver 730. Combining the outcomes from different neural networks may beimplemented using various methods. As discussed above, the predictionsmay be combined, for example, using an equal averaging method or areinforcement learning method.

In some embodiments, because the vehicle 720 is close to the object 740,the vehicle may obtain ground truth information about the object 740 andtransmit the ground truth about the object 740 to the vehicle 710. Thevehicle 710 may compare the prediction made by the vehicle 710 to theground truth received from the vehicle 720 and update the parameters ofthe neural network 712 based on the comparison.

While FIG. 7 depicts a combination of two vehicles and an edge server,any other combination may be used to classify an object using differentneural networks. For example, a combination of a ground vehicle, anunmanned aerial vehicle, and an edge server may be used to classify anobject. The ground vehicle, the unmanned aerial vehicle, and the edgeserver may store different neural networks for classifying objects,respectively. As another example, a combination of multiple vehicles anda mobile device carried by a person may be used to classify an object.The multiple vehicle and the mobile device may store different neuralnetworks for classifying objects, respectively. In some embodiments, oneor more moving vehicles and one or more parked vehicles in combined maybe used to enhance scene perception using different neural networks. Inthis example, one or more parked vehicles may operate similar to thefollowing vehicles 104 or 106 because the one or more parked vehiclesmay turn off or reduce use of some sensors (e.g., long distance radarsensors, Lidar sensors, and some cameras) and/or slow down or stopprocessing tasks (e.g., computationally intensive neural networkexecution tasks related to a particular vehicle).

It should be understood that embodiments described herein are directedto methods and systems for perceiving a scene with or without objectsexternal to a vehicle platoon using different neural networks stored indifferent vehicles of the vehicle platoon. According to the presentdisclosure, a vehicle platoon includes a lead vehicle and one or morefollowing vehicles. The one or more following vehicles may have arelatively easier driving environment as compared to the lead vehicle.The following vehicles may only be required to stay in a lane andmaintain a certain distance from the vehicle ahead. As a consequence,the following vehicles may turn off or reduce use of some sensors (e.g.,long distance radar sensors, Lidar sensors, and some cameras) and/orslow down or stop processing tasks (e.g., computationally intensiveneural network execution tasks related to the particular vehicle) andoperated mainly utilizing radar sensors and V2X communication. The savedprocessing power of the following vehicles may be redirected to helpimprove the neural network performance of the lead vehicle.Specifically, each of the following vehicles may receive raw data fromthe lead vehicle and process the raw data using its own neural networkthat is different from the neural network of the lead vehicle. Thepredictions by the neural networks of the following vehicles may betransmitted to the lead vehicle. The lead vehicle may combine itspredictions made by the neural network of the lead vehicle and thepredictions made by the neural networks of the following vehicles. Thecombined predictions may hence overall performance of the vehicleplatoon because the combined prediction may enhance the accuracy of theprediction by the lead vehicle. For example, the combined prediction mayprevent erroneous prediction by the lead vehicle due to errors insensors, the neural network, or any other processing error.

It is noted that the terms “substantially” and “about” may be utilizedherein to represent the inherent degree of uncertainty that may beattributed to any quantitative comparison, value, measurement, or otherrepresentation. These terms are also utilized herein to represent thedegree by which a quantitative representation may vary from a statedreference without resulting in a change in the basic function of thesubject matter at issue.

While particular embodiments have been illustrated and described herein,it should be understood that various other changes and modifications maybe made without departing from the spirit and scope of the claimedsubject matter. Moreover, although various aspects of the claimedsubject matter have been described herein, such aspects need not beutilized in combination. It is therefore intended that the appendedclaims cover all such changes and modifications that are within thescope of the claimed subject matter.

What is claimed is:
 1. A vehicle comprising: one or more sensorsconfigured to obtain raw data related to a scene; one or moreprocessors; and machine readable instructions stored in one or morememory modules that, when executed by the one or more processors, causethe vehicle to: process the raw data with a first neural network storedin the one or more memory modules to obtain a first prediction about thescene; transmit the raw data to a computing device external to thevehicle; receive a second prediction about the scene from the computingdevice in response to transmitting the raw data to the computing device;and determine an updated prediction about the scene based on acombination of the first prediction and the second prediction.
 2. Thevehicle of claim 1, wherein the second prediction about the scene isobtained based on the raw data and a second neural network stored in thecomputing device; and the second neural network is different from thefirst neural network.
 3. The vehicle of claim 1, wherein the firstprediction and the second prediction are prediction vectors; and theupdated prediction about the scene is determined by averaging theprediction vector of the first prediction and the prediction vector ofthe second prediction.
 4. The vehicle of claim 1, wherein the secondprediction about the scene is obtained based on the raw data and asecond neural network stored in the computing device, and wherein themachine readable instructions stored in one or more memory modules, whenexecuted by the one or more processors, cause the vehicle to: obtainground truth information about the scene; and update one or moreparameters of the first neural network based on a comparison of theground truth information, the first prediction and the secondprediction.
 5. The vehicle of claim 4, wherein the machine readableinstructions stored in one or more memory modules, when executed by theone or more processors, cause the vehicle to: in response to determiningthat the second prediction matches with the ground truth information andthe first prediction does not match with the ground truth information,update the one or more parameters of the first neural network based onparameters of the second neural network.
 6. The vehicle of claim 4,wherein the ground truth information is obtained by capturing an imageof the scene when the vehicle is within a predetermined distance from anobject in the scene.
 7. The vehicle of claim 1, wherein machine readableinstructions stored in one or more memory modules, when executed by theone or more processors, cause the vehicle to: choose the first neuralnetwork among a plurality of neural network; and broadcast informationabout the first neural network.
 8. The vehicle of claim 1, wherein thecomputing device is a second vehicle following the vehicle, and thevehicle and the second vehicle constitute a vehicle platoon.
 9. Thevehicle of claim 1, wherein the computing device is a second vehiclewithin a predetermined distance of the vehicle; and the vehicletransmits the raw data to the second vehicle via vehicle-to-vehiclecommunication.
 10. The vehicle of claim 1, wherein the computing deviceis an edge server within a predetermined distance of the vehicle; andthe vehicle transmit the raw data to the edge server viavehicle-to-everything communication.
 11. The vehicle of claim 1, whereinthe first prediction about the scene includes one of detections of oneor more objects in the scene, predicted classifications about one ormore objects in the scene, or segmentation of the scene, and the secondprediction about the scene includes one of detections of one or moreobjects in the scene, predicted classifications about one or moreobjects in the scene, or segmentation of the scene.
 12. A vehicleplatoon system comprising: a lead vehicle comprising: one or moresensors configured to obtain raw data related to a scene; a firstcontroller configured to process the raw data with a first neuralnetwork to obtain a first prediction about the scene; and a followingvehicle comprising: a second controller configured to: process the rawdata received from the lead vehicle with a second neural network toobtain a second prediction about the scene, wherein the first controlleris configured to: transmit the raw data to the following vehicle;receive the second prediction about the scene from the following vehiclein response to transmitting the raw data to the following vehicle; anddetermine an updated prediction about the scene based on a combinationof the first prediction and the second prediction.
 13. The vehicleplatoon system of claim 12, wherein the lead vehicle and the followingvehicle constitute a vehicle platoon; and the second controller isconfigured to: compare the first prediction and the second prediction;and instruct the following vehicle to opt out of the vehicle platoonbased on the comparison of the first prediction and the secondprediction.
 14. The vehicle platoon system of claim 12, wherein the leadvehicle and the following vehicle constitute a vehicle platoon; and thefirst controller is configured to: compare the first prediction and thesecond prediction; and instruct the lead vehicle to opt out of thevehicle platoon based on the comparison of the first prediction and thesecond prediction.
 15. The vehicle platoon system of claim 12, whereinthe lead vehicle obtains ground truth information about the scene; andupdates one or more parameters of the first neural network based on acomparison of the ground truth information, the first prediction and thesecond prediction.
 16. The vehicle platoon system of claim 12, whereinthe lead vehicle chooses the first neural network among a plurality ofneural network, and broadcast information about the first neural networkto the following vehicle; and the following vehicle chooses the secondneural network among the plurality of neural network in response toreceiving the information about the first neural network.
 17. Thevehicle platoon system of claim 12, wherein the second neural network isdifferent from the first neural network.
 18. A method comprising:obtaining raw data related to a scene using one or more sensors of alead vehicle; processing the raw data with a first neural network storedin the lead vehicle to obtain a first prediction about the scene;transmitting the raw data to a following vehicle; receiving a secondprediction about the scene from the following vehicle in response totransmitting the raw data to the following vehicle; and determining anupdated prediction about the scene based on a combination of the firstprediction and the second prediction.
 19. The method of claim 18,wherein the first prediction and the second prediction are predictionvectors; and wherein determining an updated prediction about the scenebased on a combination of the first prediction and the second predictioncomprises determining the updated prediction about the scene byaveraging the prediction vector of the first prediction and theprediction vector of the second prediction.
 20. The method of claim 18,further comprising: obtaining ground truth information about the scene;and updating one or more parameters of the first neural network based ona comparison of the ground truth information, the first prediction andthe second prediction.